Venue survey using unmanned aerial vehicle

ABSTRACT

A method of surveying a venue includes scanning at least a portion of the venue using an unmanned aerial vehicle having at least one scanner, converting scan data gathered by the at least one scanner into three-dimensional location data, displaying the three-dimensional location data as a three-dimensional model, analyzing the three-dimensional model, and designating portions of the three-dimensional model with semantic mapping.

RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/848,129, filed May 15, 2019, the entire content of which is herebyincorporated by reference.

FIELD

Embodiments described herein relate to surveying an environment.

SUMMARY

It would be beneficial to be able to map a venue, including a stage, inthree dimensions. The information gathered from mapping the venue can beused to create a three-dimensional (“3D”) model of the venue in adigital form. Such digital 3D models could be used in interactivedigital environments.

Currently, a user must physically measure the dimensions of a venue withmanual tools, such as a measuring tape, in order to accurately gatherdimension data of the venue. The user must then keep this dimension datawritten down or memorized in order to use it for determining lightingarrangements and the like. The dimension data is difficult to visualizewhen not viewing the venue itself, and creating a representation of thevenue, digital or otherwise, would be prohibitively time consuming dueto the number of measurements that must be made and recorded by hand. Tomake the use of such an interactive three-dimensional environmentpractical, however, the dimension data must be gathered more quickly andeasily than what can be done by hand.

For large venues in particular, some features to be surveyed and mappedmay be prohibitively high off the ground. Approaching these featureswould typically require special equipment, such as an aerial workplatform (e.g., a “cherry picker”) or scaffolding, because a laddercannot safely reach beyond a predetermined height. Personnel trained inthe use of the special equipment is then required, and risk of injury isincreased.

Accordingly, to address these and other technical problems, a system andmethod for surveying and mapping a venue, including a stage, areimplemented to gather multiple data points in a fraction of the time itwould take one or more individuals to measure the venue by hand. Thesedata points could then be imported into a specialized 3D modelingsoftware in the form of an interactive three-dimensional venue modelthat alters its display based on user input. This model is then used tocontrol one or more lights; identify areas for seating, set pieces, andother utilities; measure dimensions, clearances, and other restricted orkeep-out areas; locate rigging points; determine lighting fixturearrangement; determine lighting fixture operational capabilities;monitor airflow through the venue; measure acoustic properties of thevenue; or the like.

Methods described herein for surveying a venue include scanning at leasta portion of the venue using an unmanned aerial vehicle having at leastone scanner, converting scan data gathered by the at least one scannerinto three-dimensional location data, displaying the three-dimensionallocation data as a three-dimensional model, analyzing thethree-dimensional model, and designating portions of thethree-dimensional model with semantic mapping.

Systems described herein for surveying a venue include an unmannedaerial vehicle, at least one sensor connected to the unmanned aerialvehicle, and a controller. The controller directs the unmanned aerialvehicle about the venue, operates the at least one sensor to gathersensor data, receives a signal from the at least one sensor related tothe sensor data, translates the sensor data into three-dimensionallocation data, outputs a display of the three-dimensional location dataas a three-dimensional model, and executes semantic mapping of thethree-dimensional model.

Before any embodiments are explained in detail, it is to be understoodthat the embodiments are not limited in its application to the detailsof the configuration and arrangement of components set forth in thefollowing description or illustrated in the accompanying drawings. Theembodiments are capable of being practiced or of being carried out invarious ways. Also, it is to be understood that the phraseology andterminology used herein are for the purpose of description and shouldnot be regarded as limiting. The use of “including,” “comprising,” or“having” and variations thereof are meant to encompass the items listedthereafter and equivalents thereof as well as additional items. Unlessspecified or limited otherwise, the terms “mounted,” “connected,”“supported,” and “coupled” and variations thereof are used broadly andencompass both direct and indirect mountings, connections, supports, andcouplings.

In addition, it should be understood that embodiments may includehardware, software, and electronic components or modules that, forpurposes of discussion, may be illustrated and described as if themajority of the components were implemented solely in hardware. However,one of ordinary skill in the art, and based on a reading of thisdetailed description, would recognize that, in at least one embodiment,the electronic-based aspects may be implemented in software (e.g.,stored on non-transitory computer-readable medium) executable by one ormore processing units, such as a microprocessor and/or applicationspecific integrated circuits (“ASICs”). As such, it should be noted thata plurality of hardware and software based devices, as well as aplurality of different structural components, may be utilized toimplement the embodiments. For example, “servers” and “computingdevices” described in the specification can include one or moreprocessing units, one or more computer-readable medium modules, one ormore input/output interfaces, and various connections (e.g., a systembus) connecting the components.

Other aspects of the embodiments will become apparent by considerationof the detailed description and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

FIG. 1 schematically illustrates a system for surveying and mapping avenue in three dimensions.

FIG. 1A schematically illustrates an alternative system for surveyingand mapping a venue in three dimensions.

FIG. 2 illustrates a controller for the system of FIG. 1.

FIG. 2A illustrates a controller for the system of FIG. 1A.

FIG. 3 illustrates an unmanned aerial vehicle of the system of FIG. 1 ina venue.

FIG. 3A illustrates an unmanned aerial vehicle of the system of FIG. 1Ain a venue.

FIG. 4 illustrates and example of multiple overlapping images capturedby cameras aboard the unmanned aerial vehicle.

FIG. 5 illustrates the images of FIG. 4 stitched together to create athree-dimensional model of the venue.

FIG. 6 illustrates an example image after performing a meshsimplification.

FIG. 7 schematically illustrates an example of a semantic additionprocess.

FIG. 8 illustrates an example of a completed three-dimensional model ofthe venue.

FIG. 9 illustrates a method of operating the system of FIG. 1 or thesystem of FIG. 1A.

DETAILED DESCRIPTION

Embodiments described herein relate to gathering data points of a venuerelating to a number of different features of the venue using anunmanned aerial vehicle (“UAV”). Some embodiments further relate togathering dimension data of a venue and utilizing the dimension data tocreate an interactive three-dimensional environment to control lightsassociated with the venue in an intuitive and accurate manner. Someembodiments also relate to surveying the venue to find importantphysical features with regard to lighting capabilities and other effectscapabilities to more fully understand the venue. The informationgathered in the survey could also be loaded into the interactivethree-dimensional environment for effects experimentation.

FIG. 1 illustrates a system 100 for gathering data related to a venueincluding a stage and controlling lighting fixtures 130 according to thedata. The system 100 is provided as an example and, in some embodiments,the system 100 includes additional components. The illustrated system100 includes a user input device 105-120, a control board or controlpanel 125, at least one lighting fixture 130, at least one sensor (suchas a camera) 135 of a UAV 140 (discussed further below), acommunications network 145, and a server-side mainframe computer orserver 150. The user input device 105-120 includes, for example, apersonal or desktop computer 105, a laptop computer 110, a tabletcomputer 115, or a mobile phone (e.g., a smart phone) 120. Other userinput devices may include, for example, an augmented reality headset orglasses.

The user input device 105-120 is configured to communicatively connectto the server 150 through the network 145 and provide information to, orreceive information from, the server 150 related to the control oroperation of the system 100. The user input device 105-120 is alsoconfigured to communicatively connect to the control board 125 toprovide information to, or receive information from, the control board125. The connections between the user input device 105-120 and thecontrol board 125 or network 145 are, for example, wired connections,wireless connections, or a combination of wireless and wiredconnections. Similarly, the connections between the server 150 and thenetwork 145, the control board 125 and the lighting fixtures 130, thecontrol board 125 and the UAV 140, or the UAV 140 and the sensors 135are wired connections, wireless connections, or a combination ofwireless and wired connections.

The network 145 is, for example, a wide area network (“WAN”) (e.g., aTCP/IP based network), a local area network (“LAN”), a neighborhood areanetwork (“NAN”), a home area network (“HAN”), or personal area network(“PAN”) employing any of a variety of communications protocols, such asWi-Fi, Bluetooth, ZigBee, etc. In some implementations, the network 145is a cellular network, such as, for example, a Global System for MobileCommunications (“GSM”) network, a General Packet Radio Service (“GPRS”)network, a Code Division Multiple Access (“CDMA”) network, anEvolution-Data Optimized (“EV-DO”) network, an Enhanced Data Rates forGSM Evolution (“EDGE”) network, a 3GSM network, a 4GSM network, a 4G LTEnetwork, a 5G New Radio, a Digital Enhanced Cordless Telecommunications(“DECT”) network, a Digital AMPS (“IS-136/TDMA”) network, or anIntegrated Digital Enhanced Network (“iDEN”) network, etc.

FIG. 1A illustrates an alternative system 100A for gathering datarelated to a venue including a stage and controlling lighting fixtures130 according to the data. The hardware of the alternative system 100Ais identical to the above system 100 in every way, except the controlboard or control panel 125 is omitted. As such, the user input device105-120 is configured to communicatively connect to the lightingfixtures 130 and to the UAV 140. The connections between the user inputdevice 105-120 and the lighting fixtures 130 and the connections betweenthe user input device 105-120 and the UAV 140 are wired connections,wireless connections, or a combination of wireless and wiredconnections.

FIG. 2 illustrates a controller 200 for the system 100. The controller200 is electrically and/or communicatively connected to a variety ofmodules or components of the system 100. For example, the illustratedcontroller 200 is connected to one or more indicators 205 (e.g., LEDs, aliquid crystal display [“LCD”], etc.), a user input or user interface210 (e.g., a user interface of the user input device 105-120 in FIG. 1),and a communications interface 215. The controller 200 is also connectedto the control board 125. The communications interface 215 is connectedto the network 145 to enable the controller 200 to communicate with theserver 150. The controller 200 includes combinations of hardware andsoftware that are operable to, among other things, control the operationof the system 100, control the operation of the lighting fixture 130,control the operation of the UAV 140, receive one or more signals fromthe UAV 140, communicate over the network 145, communicate with thecontrol board 125, receive input from a user via the user interface 210,provide information to a user via the indicators 205, etc. In someembodiments, the indicator 205 and the user interface 210 may beintegrated together in the form of, for instance, a touch-screen.

In the embodiment illustrated in FIG. 2, the controller 200 would beassociated with the user input device 105-120. As a result, thecontroller 200 is illustrated in FIG. 2 as being connected to thecontrol board 125 which is, in turn, connected to the lighting fixture130 and the UAV 140. In other embodiments, the controller 200 isincluded within the control board 125, and, for example, the controller200 can provide control signals directly to the lighting fixture 130 andthe UAV 140. In other embodiments, the controller 200 is associated withthe server 150 and communicates through the network 145 to providecontrol signals to the control board 125, the lighting fixture 130, andthe UAV 140.

The controller 200 includes a plurality of electrical and electroniccomponents that provide power, operational control, and protection tothe components and modules within the controller 200 and/or the system100. For example, the controller 200 includes, among other things, aprocessing unit 220 (e.g., a microprocessor, a microcontroller, oranother suitable programmable device), a memory 225, input units 230,and output units 235. The processing unit 220 includes, among otherthings, a control unit 240, an arithmetic logic unit (“ALU”) 245, and aplurality of registers 250 (shown as a group of registers in FIG. 2),and is implemented using a known computer architecture (e.g., a modifiedHarvard architecture, a von Neumann architecture, etc.). The processingunit 220, the memory 225, the input units 230, and the output units 235,as well as the various modules or circuits connected to the controller200 are connected by one or more control and/or data buses (e.g., commonbus 255). The control and/or data buses are shown generally in FIG. 2for illustrative purposes. The use of one or more control and/or databuses for the interconnection between and communication among thevarious modules, circuits, and components would be known to a personskilled in the art in view of the invention described herein.

The memory 225 is a non-transitory computer readable medium andincludes, for example, a program storage area and a data storage area.The program storage area and the data storage area can includecombinations of different types of memory, such as a ROM, a RAM (e.g.,DRAM, SDRAM, etc.), EEPROM, flash memory, a hard disk, an SD card, orother suitable magnetic, optical, physical, or electronic memorydevices. The processing unit 220 is connected to the memory 225 andexecutes software instructions that are capable of being stored in a RAMof the memory 225 (e.g., during execution), a ROM of the memory 225(e.g., on a generally permanent basis), or another non-transitorycomputer readable medium such as another memory or a disc. Softwareincluded in the implementation of the system 100 and controller 200 canbe stored in the memory 225 of the controller 200. The softwareincludes, for example, firmware, one or more applications, program data,filters, rules, one or more program modules, and other executableinstructions. The controller 200 is configured to retrieve from thememory 225 and execute, among other things, instructions related to thecontrol processes and methods described herein. In other embodiments,the controller 200 includes additional, fewer, or different components.

The user interface 210 is included to provide user control of the system100, the lighting fixture 130, and/or the UAV 140. The user interface210 is operably coupled to the controller 200 to control, for example,drive signals provided to the lighting fixture 130 and/or drive signalsprovided to the UAV 140. The user interface 210 can include anycombination of digital and analog input devices required to achieve adesired level of control for the system 100. For example, the userinterface 210 can include a computer having a display and input devices,a touch-screen display, a plurality of knobs, dials, switches, buttons,faders, or the like. In the embodiment illustrated in FIG. 2, the userinterface 210 is separate from the control board 125. In otherembodiments, the user interface 210 is included in the control board125.

The controller 200 is configured to work in combination with the controlboard 125 to provide direct drive signals to the lighting fixtures 130and/or the UAV 140. As described above, in some embodiments, thecontroller 200 is configured to provide direct drive signals to thelighting fixtures 130 and/or the UAV 140 without separately interactingwith the control board 125 (e.g., the control board 125 includes thecontroller 200). The direct drive signals that are provided to thelighting fixtures 130 and/or the UAV 140 are provided, for example,based on a user input received by the controller 200 from the userinterface 210. The controller 200 is also configured to receive one ormore signals from the camera(s) 135 related to scan data throughcommunication with the UAV 140.

As shown in FIG. 2A and described above, the system 100A includes thecontroller 200 configured to work without the control board 125, suchthat the controller 200 is configured to provide signals to the lightingfixtures 130 and/or the UAV 140 and to receive one or more signals fromthe camera(s) 135 related to scan data through communication with theUAV 140.

FIG. 3 illustrates the control board 125, the lighting fixtures 130, theUAV 140, and the user input device 120 of the system 100 in a venue 300including a stage 305.

FIG. 3A illustrates the system 100A in a venue 300 including a stage305. As discussed above, the system 100A omits the control board 125,and the user input device 120 is configured to directly communicate withthe lighting fixtures 130 and the UAV 140.

The controller 200 receives signals from the sensors/cameras 135 aboardthe UAV 140. The controller 200 also processes the signals andinterprets the same as input data related to the dimensions and locationof the stage 305 and other features of the venue 300. The controller 200outputs a display of the venue 300 representing the dimension andlocation data to, for instance, the indicators 205. The controller 200receives user input regarding user selected lighting visuals andlighting settings to be used via, for instance, the user interface 210.The indicators 205 further display the stage in an interactivethree-dimensional environment as being altered aesthetically by theselected lighting visuals/lighting settings. The controller 200 outputsa command signal based on the entered lighting preferences. Thecontroller 200 also controls the lighting fixtures 130 according to thecommand signal either directly or through the control panel 125. Thecontroller 200 further communicates with the UAV 140 to command, forinstance, movement and activation of the UAV 140.

The UAV 140 includes, for instance, drones, remote-control helicopters,quad-copters, airplanes, balloons, and the like. Non-aerial vehiclesincluding, for instance, wheeled or tracked ground vehicles, floatingwater vehicles, submersible water vehicles, and the like can also beused as part of the system 100, 100A. The sensors/cameras 135 aboard theUAV 140 detect characteristics of the venue 300. These sensors 135include, but are not limited to, electro-optic sensors, infraredsensors, cameras, RF sensors, audio sensors, airflow sensors, airpressure sensors, temperature sensors, thermal imagers, range sensors,LIDAR sensors, GPS sensors, gyroscopes, accelerometers, motor sensors,depth cameras, orientation sensors (such as an inertial measurement unit[“IMU”]), compasses, and the like. Each sensor 135 may be fixedlyconnected to the UAV 140 or may be movable relative to the UAV 140 on,for instance, a motorized gimbal.

The UAV 140 is controlled in a variety of possible ways, includingthrough active user decisions, an autopilot control program, somecombination thereof, or the like. The control signal sent to the UAV 140is transmitted from the controller 200 or a dedicated UAV controldevice. Some embodiments include the UAV 140 including self-controlcapabilities implemented by components housed on or within the UAV 140itself.

In a fully manual mode, the user can control the UAV 140 to move inreal-time or nearly real-time with controls displayed on the user inputdevice 120. In some embodiments, the user views a live or nearly livefeed from the cameras 135 mounted to the UAV 140. In such embodiments,the user is able to pilot the UAV 140 even when a line of sight from theuser to the UAV 140 is obstructed. In the fully manual mode, the userfurther commands activation of the sensors 135 aboard the UAV 140 togather data with regard to the venue 300.

In an autopilot mode, the user places the UAV 140 somewhere to act as atakeoff and landing location, such as on the stage 305. The UAV 140 logsthis position as a reference location immediately upon startup when theuser turns on the UAV 140, in response to a command from the user viathe user interface 210, or when recognizing the UAV 140 has not movedafter a threshold period of time through data gathered by the sensors135 aboard the UAV 140. The UAV 140 automatically returns to thisreference position after completing the survey of the venue 300. In theautopilot mode, the user inputs initial information, such as the generaldimensions of the venue 300 including ceiling height and distancebetween walls representing a geofence. This information is input as partof a general flight plan for the UAV 140.

In some embodiments, the UAV 140 takes off from the stage 305 andtravels in a first direction (e.g., upwardly) until the sensors 135aboard the UAV 140 detect an obstruction (such as the ceiling or alighting fixture 130). Upon detecting the obstruction, the UAV 140further initializes other appropriate sensors 135 to fully scan theobstruction, thereby conserving battery life by not constantly utilizingall sensors 135 aboard the UAV 140. Other embodiments of the system 100,100A include all the sensors 135 aboard the UAV 140 being constantlyactive or periodically active regardless of the location of the UAV 140while the UAV 140 is on. Once an obstruction is detected and the UAV 140has finished scanning the obstruction with the sensors 135, the UAV 140moves in a second direction (e.g., horizontally) to search for anotherobstruction to scan. In some embodiments, the UAV 140 follows along thesurfaces of the first encountered obstruction and scans it with itssensors 135. In these embodiments, the UAV 140 moves in a grid-likepattern along the ceiling or in an outwardly spiraling pattern from thelocation of first encountering the obstruction. Some embodiments includesensors 135 connected to the UAV 140 via a gimbal. Such sensors 135 areadjusted such that encountered obstructions are scanned directly,obliquely, or some combination thereof with multiple passes. Otherembodiments include reference tags or other designated locations onstructures located within the venue 300. These reference tags arerecognizable through scanning done by the sensors 135 aboard the UAV140.

In either the manual control mode or the autopilot mode, the sensors 135aboard the UAV 140 scan as needed to capture the features of the venue300. In some embodiments, the sensors 135 capture data on a periodicbasis determined by a user and input via the user interface 210. Thisdata gathered during the operation of the UAV 140 as it travels aboutthe venue 300 can be used to control the UAV 140 as discussed above, butthe data may also be used as an estimation aid for offline processing ofthe data after the UAV 140 has finished traveling about the venue 300.

Other embodiments include the user deciding and controlling eachinstance of sensor activation via the user interface 210. The sensors135 capture real-world information in the venue 300 indicating the shapeand location of structures, temperature of the local areas in the venue300 (e.g., temperature information), airflow changes observed duringtravel throughout the venue 300 (e.g., airflow information), and thelike. This real-world information is stored as a series of data pointsalong with metadata for each data point, such as time, position, andorientation of the UAV 140 (or the sensor 135 on a gimbal) using, forinstance, an inertial navigation system included in the UAV 140. The useof the metadata allows the system 100, 100A to plot the data points onan absolute or relative coordinate system.

The controller 200 receives signals from the sensors 135 aboard the UAV140 and generates a 3D model from the real-world information datapoints. Some embodiments pair the data points with metadata that logs atleast one of a location of the UAV 140 and a time of the flightpath ofthe UAV 140. In some embodiments, the data points gathered while the UAV140 is traveling about the venue 300 is sent as signals to thecontroller 200 to be input in the 3D model only after the UAV 140returns to the takeoff/landing point at the end of the survey. Theseembodiments are beneficial where the data gathered by the sensors 135aboard the UAV 140 is stored locally on the UAV 140. A user then plugs awire into the UAV 140 and the user input device 120 to transmit the datamuch more quickly than by a wireless connection. Such embodiments alsoreduce the overall cost of manufacture of the UAV 140. Other embodimentsinclude the UAV 140 sending the signals representing data pointsgathered by the sensors 135 to the user input device 120 to be input inthe 3D model in real-time or substantially real-time during the surveyof the venue 300.

The generated information includes a 3D model of the venue 300, a 3Dpoint cloud, a digital surface model, a surface mesh, and the like. Aninteractive 3D model of the venue 300 is then displayed on theindicator(s) 205 using the information generated from the receivedsensor data. In some embodiments, the controller 200 renders theinteractive 3D model altogether. Other embodiments including the UAV 140transmitting data gathered by the sensors 135 during the operation ofthe UAV 140, either in real-time or near real-time, allow for thecontroller 200 to render the interactive 3D model piece by piece. Forinstance, depth cameras 135 on the UAV 140 capture images that are usedto form and build up a 3D model of the venue 300 as the UAV 140 movesabout the venue 300.

In some embodiments, depth information captured by the depth cameras 135on the UAV 140 is gathered in a manner including use of, for instance,time of flight, structured light, or stereo images. In some embodiments,the 3D model and the UAV autopilot mode work in tandem to track theposition of the UAV 140 in relation to the 3D model being created of thevenue 300. This combination of tracking and 3D modeling is known assimultaneous localization and mapping (“SLAM”). As shown in FIG. 4,multiple overlapping images are captured by the cameras 135 aboard theUAV 140. These images are stitched together (FIG. 5) to create the 3Dmodel of the venue 300. The SLAM algorithm used to track and control theUAV 140 as it travels about the venue 300 also gathers data thatprovides an initial estimate of the orientation and position of the UAV140 relative to the venue 300 for later offline processing of all thedata gathered by the sensors 135 aboard the UAV 140. This initialestimate allows the calculations to approximate an error function andconverge on a best estimation of the model of the venue 300.

The 3D model and the autopilot function of the UAV 140 further worktogether by providing feedback to the user in the form of thepiece-by-piece creation of the 3D model. The in-progress 3D model isdisplayed on the user input device 120, for instance, allowing the userto observe the quality and progress of the 3D model. If portions of thevenue 300 shown in the 3D model do not match what the user expects tosee based on his own visual inspection of the venue 300, the user mayflag the problem area in the 3D model via input through the userinterface 210. This problem area designation received by the controller200 allows the controller 200 to send data to the UAV 140 in the form ofupdated autopilot flight control signals. The flight control signals aresent to the UAV 140 to move the UAV 140 to the portion of the real worldvenue 300 corresponding to the problem area and to control the UAVsensors 135 to recapture data regarding that portion. The problem areamay be due to any number of issues including, for instance, out of focusimages, insufficient light, missed data points, signal interference, andthe like. In some embodiments, the inspection of the 3D model todetermine problem areas is run automatically by the controller 200 as apart of the 3D model creation instead of requiring the user's input. Thecontroller 200 could recognize images of poor quality as beinginsufficient for the 3D model based on an evaluation of the continuityof a given image with adjacent images for stitching together the images(via e.g., random sample consensus and/or iterative closest point),sufficient overlap between images, distortion, and the like.

In embodiments including the controller running both the autopilotfunction of the UAV 140 and the creation of the 3D model simultaneously,the 3D model informs the autopilot routine of the UAV 140 when thewindow of a current data set is partially overlapping with that of aprevious window already integrated into the 3D model. In this manner,the system 100, 100A is able to move forward through the surveyoperation throughout the venue 300 without depending on incrementalmovements alone, and the system 100, 100A recognizes when an area iscomplete or even when the entire survey operation is complete withoutinput from a user.

In addition to the above described functionality, the controller 200 isfurther configured to convert a pointcloud of data points gatheredduring the survey operation into a mesh. This mesh runs through a meshsimplification algorithm within the 3D model to look for surfaces thatmatch known descriptors including, for instance, planes, spheres, andcylinders. The simplified mesh (shown in FIG. 6) then runs through, forinstance, random sample consensus mapping, iterative closest pointmapping, feature based mapping, or the like of known object descriptors.As shown in FIG. 7, the mapping software adds semantics to the map toidentify, for instance, chairs, trusses, tables, lighting fixtures 130,and the like.

The data gathered and interpreted by the controller 200 via the UAV 140allows for mapping of the entire venue 300 including mounting locationsfor lighting fixtures 130, lighting fixtures 130 themselves (includingposition and type of lighting fixtures 130), obstructions for lightingfixtures 130 with regard to lighting portions of the stage 305, and thelike.

With the completed 3D model, a virtual camera can observe the 3D modeland feed the views into a neural network algorithm (such asConvolutional Neural Networks with the TensorFlow framework). Thisalgorithm can be applied to expand the semantical mapping of the itemsin the 3D model. The neural network algorithm is adapted to recognizemany theatre specific features, common lighting plot features, how eachlighting fixture 130 in a venue 300 is used, and the like.

As shown in FIG. 8, the completed 3D model can be used to measurefeatures such as height clearances, mounting locations for lightingfixtures 130, floor area of the stage 305, or the like digitally insteadof by hand in the venue 300 itself. Further, data gathered during theabove-described survey operation can be analyzed to identify variationin internal control loops of the UAV 140. These variations can be usedto identify and measure any external disturbances on the flight of theUAV 140. For indoor venues 300, the disturbance can be attributed toairflow in the venue 300 instead of wind due to weather. Thisinformation can be utilized for determining proper placement for smokeand haze machines in the venue 300.

FIG. 9 illustrates a method 900 of surveying the venue 300 with the UAV140 to create a 3D model. The method 900 includes placing the UAV 140 ata known takeoff point (step 905). This takeoff point is on the stage305, for instance. The UAV 140 calculates its location during theremainder of the method 900 based on this takeoff point. The method 900further includes initiating a survey operation of the UAV 140 (step910). This step includes starting the UAV 140 and moving the UAV 140 toa location to begin scanning the venue 300. The venue 300 is scanned inits entirety with sensors 135 using the UAV 140 (step 915). The datagathered by the sensors 135 aboard the UAV 140 during the surveyoperation is then processed by the controller 200 (step 920). Thecontroller 200 generates and outputs a display of the 3D model on, forinstance, the user input device 120 (step 925). Either automatically orupon initialization by the user, the method 900 continues by analyzingthe 3D model for relevant features (step 930). This step, in someembodiments, utilizes control log data and other sensor data gatheredduring the method 900. The features discovered in the analysis of the 3Dmodel include, for instance, measurements of the venue 300 and objectstherein, location of airflows, temperatures of lighting fixtures 130,and the like. The method 900 further includes semantic mapping of theanalyzed 3D model (step 935). The 3D model, in some embodiments, isupdated to include labels for the components of the 3D model viewableupon selection, for instance. Other embodiments simply log the semanticinformation for data gathering purposes.

Some embodiments of the method 900 further include receiving user inputsfrom a user regarding lighting visuals and/or lighting settings for thelighting fixtures 130 via, for instance, the user input device 120 (step940). The user input, in some embodiments, includes an interaction bythe user with features of the display of the interactivethree-dimensional environment. These interactions include, for instance,directing beams of light from a lighting fixture 130 to a destinationindicated in the interactive three-dimensional environment, changing abrightness or color of a light projected by a lighting fixture 130 byadjusting dials or other controls displayed in the interactivethree-dimensional environment, indicating a light destination byselecting a location on the stage 305 displayed within the interactivethree-dimensional environment, or the like. Once the user input has beenreceived (step 940), the method 900 further includes controllinglighting fixtures 130 to move or otherwise adjust in order to match thedesired results indicated by the user through the user input (step 945).This step is accomplished by exporting a command signal (or a commandstring) to the lighting fixtures 130 either directly or indirectlythrough the control board 125 to control the lighting fixtures 130.

Thus, embodiments described herein provide methods and systems forsurveying a venue, logging information related thereto, and controllingat least one lighting fixture according to the information. Variousfeatures and advantages of some embodiments are set forth in thefollowing claims.

What is claimed is:
 1. A method of surveying a venue, the methodcomprising: scanning at least a portion of the venue using an unmannedaerial vehicle having at least one scanner; converting scan datagathered by the at least one scanner into three-dimensional locationdata; displaying the three-dimensional location data as athree-dimensional model; analyzing the three-dimensional model; anddesignating portions of the three-dimensional model with semanticmapping.
 2. The method of claim 1, further comprising displaying thethree-dimensional model as an interactive three-dimensional environment.3. The method of claim 2, further comprising: receiving a user input asan interaction with the interactive three-dimensional environment; andchanging a setting of at least one lighting fixture in response to theuser input.
 4. The method of claim 1, wherein: converting the scan dataincludes determining a shape and a location of one or more lightingfixtures; and displaying the three-dimensional location data includesdisplaying the shape and the location of the one or more lightingfixtures.
 5. The method of claim 4, wherein: analyzing thethree-dimensional model includes identifying a type of the one or morelighting fixtures; and designating portions of the three-dimensionalmodel with semantic mapping includes designating the type of the one ormore lighting fixtures.
 6. The method of claim 1, wherein: scanning thevenue includes measuring temperature information of local areas in thevenue; and designating portions of the three-dimensional model withsemantic mapping includes designating portions of the three-dimensionalmodel with the temperature information.
 7. The method of claim 1,wherein: scanning the venue includes measuring airflow information oflocal areas in the venue; converting the scan data includes determiningairflow direction and airflow intensity of the local areas in the venue;and designating portions of the three-dimensional model with semanticmapping includes designating portions of the three-dimensional modelwith the airflow information.
 8. The method of claim 1, wherein:analyzing the three-dimensional model includes identifying a seatingarea in the venue; and designating portions of the three-dimensionalmodel with semantic mapping includes designating the seating area. 9.The method of claim 1, wherein: analyzing the three-dimensional modelincludes identifying a set piece; and designating portions of thethree-dimensional model with semantic mapping includes designating atype of the set piece.
 10. The method of claim 1, wherein: analyzing thethree-dimensional model includes identifying a restricted area; anddesignating portions of the three-dimensional model with semanticmapping includes designating the restricted area.
 11. The method ofclaim 1, wherein: analyzing the three-dimensional model includesidentifying a rigging point; and designating portions of thethree-dimensional model with semantic mapping includes designating therigging point.
 12. The method of claim 1, wherein: scanning the venueincludes measuring acoustic properties of local areas in the venue; anddesignating portions of the three-dimensional model with semanticmapping includes designating portions of the three-dimensional modelwith the acoustic properties.
 13. A system for surveying a venue, thesystem comprising: an unmanned aerial vehicle; at least one sensorconnected to the unmanned aerial vehicle; and a controller configuredto: direct the unmanned aerial vehicle about the venue; operate the atleast one sensor to gather sensor data; receive a signal from the atleast one sensor related to the sensor data; translate the sensor datainto three-dimensional location data; output a display of thethree-dimensional location data as a three-dimensional model; andexecute semantic mapping of the three-dimensional model.
 14. The systemof claim 13, wherein the three-dimensional model includes an interactivethree-dimensional environment.
 15. The system of claim 14, furthercomprising: at least one lighting fixture; and wherein the controller isfurther configured to: receive user input as an interaction with theinteractive three-dimensional environment; and change a lighting settingof the at least one lighting fixture in response to the user input. 16.The system of claim 13, wherein the at least one sensor includes acamera.
 17. The system of claim 13, wherein the at least one sensorincludes a temperature sensor.
 18. The system of claim 13, wherein thecontroller is further configured to automatically direct the unmannedaerial vehicle about the venue in an autopilot mode.
 19. The system ofclaim 13, wherein the controller is further configured to track a flightpath of the unmanned aerial vehicle while operating the at least onesensor.
 20. The system of claim 19, wherein the controller is furtherconfigured to determine airflow characteristics of the venue based onthe flight path and the sensor data.